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Abstract. We provide a Hop-Lax formula for variational problems with non-constant 
discount and deduce a dynamic programming equation. We also study some regularity 
properties of the value function. 



1. Introduction 

We establish a Hopf-Lax formula for the Cauchy problem 

f -vt(x, t) + f{-v x ) + p(t)v(x, t) = 0, in R™ x (0, T); 
\ v = g, on R n x {t = T}, 

involving a Hamilton-Jacobi equation with a linear dissipation term, p(t)v(x,t), and a 
terminal condition at time t = T. The function / : R n — ► R is assumed to be convex and 
of class C 2 , p : [0, T] — ► (0, 1] is continuous, and g : R n — > R is globally Lipschitz. The 
formula is 

v(x,t) = min d t (s)£(L(dt 1 (s)p)) ds + dt(T)g (x + J o(dt 1 (s)p) ds 

where I is the convex conjugate of /, i = (V^)" 1 , and dt(s) = exp (— p(r) dr) . The 
formula represents a Lipschitz solution that satisfies the Cauchy problem almost everywhere. 

The classical Hopf-Lax formula applies to the case p = , and was given by Lax in [6], 
for n = 1. It was extended later to general n by Hopf in [5]. Further generalizations have 
maintained p = but have considered functions f(t,—v x ) also depending on time, [S]; or 
functions f(v,—v x ) depending on both v and v x , [2], with some additional requirements. 
The case we analyze in this paper is not covered in any of these previous works. 

Actually, we find a Hopf-Lax formula that applies to more general Hamilton-Jacobi 
equations associated to the calculus of variations problems with variable discount. 

These problems arise quite naturally in models of economics. Consider for instance the 
following problem: an agent optimally chooses a consumption path of a given good, with 
the aim of maximizing his/her satisfaction. This is measured by an utility function of 
consumption, £(u), along a given time interval, [0, T\. It is customary in the literature to 



Date: April 21, 2009. 



1 



2 



JUAN PABLO PJNCON-ZAPATERO 



postulate concavity in the preferences of the agent, and to suppose that he/she is impatient, 
in the sense that the value of the utility attained today is higher than the utility attained 
tomorrow. This is the meaning of introducing a discount factor or impatience rate in the 
preferences of the agent. 

Empirical studies suggest that people are more impatient about choices in the short run 
than in the long run, implying that the discount rate applied to current choices is higher 
than the one applied to far-in-the future choices. Thus, the discount factor should be taken 
to be non-constant. Several papers have considered the non-constant discount case: see e.g. 
PP, [1] or [?|*[ In [I], the optimal growth model with time-varying discount is considered, for 
a particular class of utility functions. A general problem, with infinite horizon, is analyzed 
in [1], whereas [7] considers the finite horizon case with fixed or variable terminal time. 
The last two papers use discretization and passage to the limit to find a Hamilton-Jacobi 
equation that involves not only the unknown value function, but also a non-local term 
involving integration along the unknown optimal solution. We provide conditions so that 
the Hamilton-Jacobi equation involves only the derivatives of the value function and find 
the dynamic programming equation by direct methods. 

Given the significance of the non-constant discount preference rate in economics, it is of 
interest to analyze in more detail this type of variational problems. First, deriving a Hopf- 
Lax formula for the solution of the variational problem (Section [3]); second, establishing a 
modified dynamic programming equation, more amenable than the one found in previous 
papers (Section and third, studying the regularity of the value function (Section [5]). 

2. Variational problem with discount 
We follow the presentation in [3]. Let the value function 

(2) v(x,t)= inf \ [ T d t (s)£(y( S ))d S + d t (T)g(y(T))\ , 

where 

AC^t = {y '■ [t,T] — > W 1 : y = y(s) absolutely continuous, y(t) = x}. 

A typical element of this set will be called an arc. We will impose the following conditions. 

£(u) 

Al: i : W 1 — > K is C 2 , strictly convex, and lim -p-p = oo; 
A2: g is globally Lipschitz in W 1 ; 

A3: d : [0, T] x [0,T] — > (a, 1], with a > 0, is Lipschitz continuous with dt(t) = 1 for 
each t. 



We consider only papers on continuous time. 
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A straightforward interpretation of ([2]) has been done in the Introduction: a single agent, 
with time-varying preference rate, chooses optimally along time. In another reading, there 
is a continuum of agents, each one labelled by t € [0, T]; each agent (or generation t) applies 
a possibly different discount factor, dt, in the calculation of the utility flow from t onwards. 
At time T the optimization process finishes and agent T derives utility g(y(T)) (or "scrap 
value"). The aim of each generation is to maximize the total discounted utility. In this 
process, the t-generation is not so much concerned with the consumption of the future 
generations as it is with respect its own consumption. 

A common specification of dt(s) is 



where p € L°°([0,T]). In this case dt(s) is Lipschitz in (t, s), which is the present value 
at time t of one unit of utility at time s > t. The rate of discount is p, and most often it 
is considered constant. Other popular discount factors are those that depend only on the 
elapsed time, dt(s) = 9(s — t) for s > t, through a scalar function 9, with 6(0) = 1. As will 
be seen in Section U the shape of the discount factor has a major effect in the structure of 
the dynamic programming equation. 

Let us define t = (V£) , the inverse of W. Notice that by Al, both V£ and t are 
continuous, and suprajective. We also consider £*{p) = sup ugR n{p ■ u — £(u)}, the Legendre 
transform of I. Finally, let the t-Hamiltonian 



Throughout the paper, V denotes the gradient of a real function, and V 2 the Hessian 
Matrix. For a vector function, V denotes the Jacobian matrix. 



A Hopf-Lax formula describes an infinite dimensional variational problem as a finite 
dimensional one. In the present case, the formula is a bit more involved than in the non- 
discounted case, due to the non-autonomous term dt(s). Notice also that the problem at 
hand is different from the one with a non-autonomous £(s,y(s)), because the current date 
t enters into the definition. 



(3) 




H t (s,u,p) =p-u- d t {s)£(u). 



3. Hopf-Lax formula 
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Given t G [0, T], t < s < T, x, a G R n , consider 

Ut, a (s) = L(dt\s)Ve(a)), fa 1 = l/d t ) 
Y t , x , a {s) = x + f i{d;\r)Vi{a)) dr, 



V(x, t, a) = ^ d t {s)l{U t , a {s)) ds + d t (T)g(Y t>x>a (s)) . 

Notice that Yt )X)a {s) is absolutely continuous and Y x t a {t) = x, thus it is an admissible arc, 
i.e. it belongs to AC^. Observe also that Y X)tt a(s) = Ut : a{s). 

We establish the following lemma to facilitate posterior quotation. It is a consequence of 
assumption Al. 

Lemma 3.1. For x G W 1 , t G [0, T), s >t, the mappings a t— > Ut t a(s), ct i— > Yt )X) a{s) are of 
class C 1 and suprajective. 

Theorem 3.1. (Hopf-Lax formula with discount). If x £ M n and < t < T, then the 
value function v = v(x, t) of the minimization problem ([5]) is given by 

(4) v(x,t) = min jjf d t (s)£(L(d^ 1 (s)p)) ds + d t {T)g(x + jf (s)p) ds 



Proof. 1. For any a G 



and so 



t) < / dt(s)e(Y ttX>a (s)) ds + d 4 (T) 5 (F tia:iCl (T)) = V(x, t, a), 



v(x,t) < inf V(x, t,a). 

o£R n 



2. On the other hand, for an arbitrary function y(s), t < s < T, with y(t) = x, let a be 
such that 

Y t , x , n (T) = y(T). 

This is possible by Lemma [3TT1 For each t, s, p, the Hamiltonian H t (-, u, •) is concave, thus 
for any a 

(5) H t (s,U t , a (s),Ve(a)) > H t (s,y(s),V£{a)), 



since 

dH t (s,u,V£(a)) 



du 



= 0. 

U=Ut,a{s) 
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Let a = a defined above. Integrating ([5]) between t and T and rearranging terms we get 
T d t (s)£(U m {s))) ds < ^ d t (s)£(y(s)) ds + V£{a) j\u m (s)) - y(s)) ds 

T d t {s)£{y{s)) ds + W(a) F {Y x , m {s)) - y{s)) ds 

T 

d t {s)l{y{s))ds, 

because Y x , m {t) = x = y{t) &n&Y x ^{T) = y{T). Adding d t {T)g{Y x ^{T)) = d t {T)g{y{T)) 
to both terms of the above inequality we get that, for any arc y{s), there exist some a such 
that 

V(x, t,a) < ^ d t (s)£(y(s)) ds + d t (T)g(y(T)). 

Thus inf aS Rn V(x,t,a) < v(x,t). Hence, inf Qg Rn V(x,t,a) = v(x,t). Finally, observe that 
minimization with respect to a is equivalent of minimization with respect to p = W(a). 

3. The infimum is in fact attained, since the function V(-,-,a) is continuous and inf- 
compact. Indeed, lim^i^go |a|~ 1 y(x, t, a) = oo due to the assumptions A1-A3 and Lemma 
I3T1 □ 

Define A{x, t) = argmin QgR n V(x, t, a). Since lim^i^oo |a| _1 y(x, t, a) = oo, A is compact 
valued and upper semicontinuous correspondence. 

The following corollary is along the lines of the above proof. 

Corollary 3.1. If x G M n and < t < T, then for any selection a(x,t) S A(x,t), the arc 
Y x ,t,a(x,t)( s ) i s a solution of problem ([2]). 

Remark 3.1. When dt(s) = 1 for all < t < s < T , reduces to the classical Hopf-Lax 
formula 

" (i - t) =s.{ (T - t K^) +9(o) }- 

For a locally Lipschitz function /, lip(/) will denote the Lipschitz parameter of / in a 
given compact set K, and bound(/) will denote a bound of |/| in that set. Notice that 
under our assumptions / = i, d, d~ l are locally Lipschitz. 

Theorem 3.2. (Lipschitz continuity). The value function v is locally Lipschitz continuous 
in W l x [0, T] and 

v = g onW 1 x{t = T}. 
Proof Let x, x G K C W 1 with K compact, and t, i G [0, T) and a G W 1 . 
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1. Let s € [0, T). Let us proceed to establish three Lipschitz estimates. 

\U { Js) - U t>a (s)\ < hplOV^a)!^ 1 ^) - d^(s)\ 

< lip(0V£(a)lip(d- 1 )|t-t| = C\i-t\. 



\ Y lx,a( S ) -Yt,x, a (s)\ <\X~X\ + 



U ia (s)ds- / UtM\ds 



t Jt 
T 



I: VI 



<\x-x\ + \U { (s) - U t , a (s)\ ds + / \U iMtCe (s)\ds 

JtVt JtM 

< \x-x\ +TC\i-t\ +bound(U)\i-t\ 
= \x -x\ + C\t-t\. 



\di(T)g(Y tAa {T)) - d t (T)g(Y tjXja (T))\ < \d { (T) - d t (T)\\g(Y tAa (T)\ 

+ d t (T)\g(Yf Aa (T))-g(Y t>x>a (T))\ 

(6) < lip(d)bound(sO|£-t| 

+ lip(g) bound(d) (\x - x\ + C\i - t\) 
= C(\x -x\ + \t-t\). 

In the above, we have used the same C to denote several constants. 

3. Choose a £ A(x,t). Then, by definition of v and estimate © 

v(x, t) — v(x, t) < V(x, t, a) — V(x, t, a) 

= d { (T)g(Y {Aja )-dt(T)g(Y t ^ a ) 
< C(\x-x\ + 

Reversing the role of (x,t) and (x, t) we get the desired Lipschitz property. 

4. Now, let x € W 1 , t < T and define S t = d t {s)ds. Choose a € W 1 such that 
ft' Ut s a(s) ds = 0; this is possible by virtue of Lemma [37TI Let b = max se u,T] |L^ jQ! (s)|, and 
let bound(-£) be a bound of \£\ in [—6, b\. Then, 



(7) 

Moreover, 



T 

v(x,t)< I d t (s)£(U tta (s))ds + d t (T)g(x) < bound(£) S t + d t (T)g(x). 



T 



v(x,t) > d t (T)g(x) + min (-lip(^) / U t , a {s)ds + dt(s)£(U t>a (s)) ds 



> d t {T)g{x) + St min {- lip^)^" 1 F U t , a (s) ds + £ ( 5^ C . 



dt(s)U t , a (s) ds 
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by Jensen's inequality. Now, notice that for any a € M n 
jj 'Ut a(s) ds 

—j, ■ ► 1 as t — > T (componentwise) 

f t d t (s)U t , a (s) ds 

thus, for every t close enough to T, there exists e > such that 

v(x,t) > d t (T)g(x) + 5 t min ( - lip(ff)(l + e)^ 1 / d t {s)U %ol {s) ds 

oeR™ L Jt 

8t l J d t {s)U t . a {s)ds 

= d t {T)g{x) — 5 t max max ^z<5 t _1 J d t (s)Ut jOC (s) ds 

5t l J d t (s)U t , a (s) ds 
where B = [— lip(£)(l + e),lip(£)(l + e)]. Then 

(8) v(x,t) > d t (T)g(x) - 5 t max t(z), 

Z e[-lip(€)(l+ £ ),lip(*)(l+e)] 

since a — ► dt(s)Ut ja (s) ds is suprajective. Thus, by © and (jSJ) 

\v(x,t) - d t (T)g(x)\ <C5 t 

for an appropriated constant C. Given that dt(T) tends to 1 and 6t tends to as t — > T, 
we are done. □ 

4. Dynamic programming equation 

For any y £ AC x j and t < r < T, let Y T denote an optimal arc from initial condition 
(y{T~),T), that is, 

^t(^) ^j/(T),r,a(j/(r),r) (^)j 

which exists by Corollary 13. 1[ 

Consider for t < t <T the function 

t, r) = - d r ( S ))^(F T (s)) (is + (d t (T) - d T (T))g(Y T (T)) 



Lemma 4.1. For every initial condition (x,t), admissible arc y £ AC x j and t < t < T , 
we have 

(9) v{x,t)<J d t (s)l(y(s))ds + v(y(T),T) + W(x,t,T). 
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Proof. Let y 6 AC Xj t be fixed but arbitrary. If r = T, then y(T) = Yt(T) and v(y(T),T) = 
d T {T)g{y(T))=g{Y T {T)). Then © reduces to v(x, t) < f f T d t (s)£(y(s)) ds + d t (T)g(y(T)), 
which is true by the definition of v. Now, suppose r < T. Let a(y(r),r) G A(y(r),r). 
Then, by Corollary 13.11 

d r (s)£(F r ( S )) + d T (T) ff (y r (r)) = v(y(r),r). 



Let us define the admissible arc y G AC^t by 

f if t < s < r; 



We have 



F r (s), ifr<s<T. 



v(x,t) < [ T d t (s)£^(s))ds + d t (T)g(y(T)) 

d t (s)£(y(s)) ds + £ d T (s)£(Y T (s)) ds + d T (T)g(Y T (T)) 
+ f T (d t (s) - d T (s))£(Y T (s)) ds + (d t (T) - d T (T))g(Y T (T)) 



d t (s)£(y(s))ds + v(y{T),T) 



T 



+ / (dt(s) - d T (s))£(Y T (s)) ds + (dt(T) - d T (T))g(Y T (T)). 



Corollary 4.1. (Dynamic Programming). For every initial (x,t) and t < r <T 



□ 



(10) v(x,t)= min \ d t (s)£(y(s))ds + v(y(T),T)\ +W(x,t,r). 

Proof. In fact ([9|) is an equality since an optimal arc is attained for every initial condition 
x.t. by Corollary 13.11 □ 

Now consider the function 

-^(s)£(U t , a (s))ds-^(T)g(Y xAa (T)). 

The (generalized) dynamic programming equation is as follows. It could be obtained for a 
more general optimal control problem with some additional assumptions. 

Theorem 4.1. (Dynamic Programming Equation). Suppose that for every t < s < T , 
dt(s), (d/dt)dt(s) are continuous in t and summable in s. Let (x,t) be a point at which the 
value function v is differentiable. Then: 

(12) - v t (x, t) + £*(-v x (x, t)) + w(x, t, a{x, t)) = 0, in R n x (0, T). 
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Proof. 1. By Lemma SU if t + h < T 

rt+h 

v(x,t)-v{y(t + h),t + h)) < J d t (s)£(y(s))ds 
(13) + [ T (dt(s) - d t+h (s))£(Y t+h (s)) ds 

Jt+h 

+ (d t (T)-d t+h (T))g(Y t+h (T)), 

for any y £ AC X)t . 

2. The correspondence A is compact valued and upper semicontinuous, hence we can 
assume lim h _ >0 + a(y(t + h),t + h) £ A(x,t); we denote the limit by a(x,t). By continuity 
lim ft _ + Y t+h{s) = Y t (s), and lim ft _ + Y t+h(s) = Y t (s). Then, 

dd t 



and 



\\m + h-\d t {T)-d t+h {T))g(Y t+h {T)) = —^(T)g(Y t (T)), 



lim hT x l T (d t (s)-d t+h (s))e(Y t+h (s))ds = - ( T ^( s )£(Y t (s))ds. 



fc.-*o+ .A dt 

3. Taking limits in (fT3 



^lim + /i -1 t) - t>(y(i + fr), t + h)) < lim + h' 1 J d t (s)£(y(s)) ds 

- lim h~ x W{x,t,t + h) 

h — >0 

for every y € AC Xi j. This yields 

—vt(x, t) — v x (x, t) ■ u — £{u) + w(x, t, a(x, t)) < 

for every u 6 R n . Recalling the definition of £* , this is equivalent to 

— t) + £*(— v x (x, t)) + ui(:r, t, a(x, t)) < 0. 

4. To prove the equality, we use the same argument. Notice that equality holds in (|13p for 
y(s) = Y t (s). □ 

Remark 4.1. // ^(s) = p(t)d t (s) for some continuous function p, then equation (jlip 
gives w(x,t,a(x,t)) = p(t)v(x,t) hence, (|12|) tafces the form of a Hamilton-Jacobi equation 
with a dissipation term 

-v t (x, t) + £*(-v x (x, t)) + p(t)v(x, t) = 0, in R n x (0, T). 



27ms happens if and only if ([3|) holds, since we are assuming dt(t) = 1 for each t. 
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In the general case, the dynamic programming equation (|12p has a complicated structure. 
Indeed, the optimal arc itself enters the formulation as a non-local term, thus the applica- 
bility of the equation should be taken with caution. In contrast, the solution given in (J4|) is 
simpler. This stresses the usefulness of having a Hopf-Lax formula at hand. Nevertheless, 
we can give a more amenable form to the dynamic programming equation, close to classical 
standards, when assuming that both the value function and function g are differentiable. 
This is the content of the next theorem. 

Theorem 4.2. With the same assumptions as in Theorem \4-l\ assume further that \7 2 £(u) 
is definite positive for every u € M n and that g is differentiable; then, the dynamic program- 
ming equation (|12p is 

(14) - v t (x, t) + £*(-v x (x, t)) + w(x, t, i(-v x (x, t))) = 0, in M. n x (0, T). 

Proof. Since we are supposing v is differentiable, the envelope theorem applied to gives 

v x (x, t) = d t (T)Vg (lt )Z , a ) . 
On the other hand, a is an irrestricted minimum of V, hence 
= V a (x, t, a) 

= (ve(a)+dt(T)Vg(Y t ^ a fj ^jT d^ 1 {s)V l^ 1 {s)V £{a)) ds^j V 2 £(a) 
= (ve(a)+v x (x,t)\ (J^ dt\s)Vi(dt l (s)V£(a))ds^V 2 £(a). 

Since V 2 £ has maximal rank and Vi(-) = (V 2 £(-)) _1 , the gradient of V with respect to a is 
the null vector only if V£{a) = —v x (x,t) and then a(x,t) = i(—v x (x,t)) at points of differ- 
entiability of v (incidentally, this shows that a must be unique at points of differentiability 
of v). Plugging this value for a into w(x,t,a) we reach the expression for the dynamic 
programming equation asserted in the theorem. □ 

5. Regularity of the value function 

By Rademacher's Theorem, a locally Lipschitz function is almost everywhere differen- 
tiable. Thus, by Theorem 13.21 the value function v, which is characterized by ([!]) also 
satisfies the dynamic programming equation almost everywhere. Summarizing: 



Theorem 5.1. With the same assumptions as in Theorem the function v defined by 
the Hopf-Lax formula is the value function ([2]), which is locally Lipschitz continuous in 
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n x [0,T), and solves the terminal value problem (in a generalized sense) 
\ -v t + £*(-v x ) + w(x,t,i(-v x )) = 0, a.e. € M. n x (0, T); 



(15) 



v = g, onW n x{t = T}. 



In the conditions of the above theorem, for the particular case of the Hamilton-Jacobi 
equation with dissipation we have 

Corollary 5.1. Function v given by (j3J) with dt(s) = exp (— p(r) dr) , t < s < T is 
locally Lipschitz continuous in M. n x [0,T), and solves the terminal value problem ([1]). 

Now we establish some results on the smoothness of the value function. 



Theorem 5.2. With the same assumptions as in Theorem suppose further that g is 
convex. 

(1) If g is of class C , then the value function v is differentiable in R" x (0,T) and the 
minimizer a is continuous. 

(2) If g is of class C 2 , then the value function v is also of class C 2 in W 1 x (0,T) and 
the minimizer a is of class C 1 in W 1 x (0,T). 

Proof. The minimizers a in (JH) satisfy 

(16) V%) + d t (T)Vg (Y tyXya {T)) = 0, 



as shown in the proof of Theorem 14.21 

1. For x,t fixed but arbitrary, the mapping a i— > X7£(a) + dt{T)Vg (Yt,x,a(T)) is strictly 
monotone due to the convexity of g and the strict convexity of £, thus a(x, t) is unique; as a 
correspondence, {a(x,t)} is upper semicontinuous thus, as a function it is continuous. The 
uniqueness of a leads to the differentiability of the value function, by Danskin's Theorem. 

2. The derivative of the L.H.S. of (|16p with respect to a is 

/ + d t (T)V 2 g(Y x ^ a (T)) ^ dt\s)Vi{dt X {s)Vl{a)) ds^j V 2 £(a), 

with / being the identity matrix. Given our assumptions, this vector has norm > 1 hence, 
(|16p locally defines a(x,t) of class C . This function is defined globally since the mapping 
a i — ► X7£(a) + dt(T)Vg (Yt,x,a{T)) is proper because 



lim V£(a) + d t (T)Vg (Y t ^ a (T)) = ±oo. 

a— >±oo V / 

By the envelope theorem, v x (x,t) = V ' g{Y x ^^ x ^{T)) is of class C 1 hence, by the dynamic 
programming equation (|14p . vt is also of class C . □ 

Finally, a result concerning the monotonic behavior of a in the scalar case. 
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Theorem 5.3. With the same assumptions as in Theorem \4-l\ with n = 1, assume further 
that g is convex and of class C 2 . 

(1) For each time < t < T, there exists for all but at most countably many values of 
x € M a unique point a(x,t) where the minimum in is attained. 

(2) The mapping x i— > a(x, t) is nondecreasing. 

Proof. Since v is locally Lipschitz, it is differentiable almost everywhere, thus the minimum 
a is unique almost everywhere. On the other hand, the crossed derivative of V(x, t, a) with 
respect to x and a is 

dt{T)l"{a)g"{Y t>x>a ) ^ d^ 1 (s)l' (d^ 1 (s)V £(a)) ds > 0. 

Hence, (x,a) i— » V(x,t,a) is supermodular, and by Topkis' Theorem, [8], A(x,t) is a 
nonempty compact sublattice which admits a lowest element, which we denote again by 
a(x,t), satisfying a(x2,t) > a(xi,t) whenever x^ > x±. Then the mapping x i— > a(x,t) is 
non-decreasing and thus continuous for all but at most countably many x. □ 
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